1. Field of the Invention
Various optical apparati such as microscopes, endoscopes, telescopes, cameras etc. support viewing or analyzing the interaction of light with objects such as planets, plants, rocks, animals, cells, tissue, proteins, DNA, semiconductors, etc. Accordingly, reflected and/or light emitted from the interaction of objects with light, may provide multi-band spectral images yielding useful information related to physical structure (morphological image data) and/or spectral image information related to the chemical make-up, sub-structure and/or other characteristics related to the target object. These light emission images, such as luminescence or fluorescence, may also provide a means to assess endogenous chemicals or exogenous substances such as dyes employed to enhance visualization, drugs, therapeutic intermediaries, or other agents.
In the field of medical imaging and more particularly endoscopy, reflected white light, native tissue autofluorescence, luminescence, chemical emissions, near-IR, and other spectra provide a means to visualize tissue and gather diagnostic information. In addition to visualization of tissue morphology the interaction of light in various parts of the electromagnetic spectrum has been used to collect chemical information. Three general real-time imaging modalities for endoscopy that are of interest include white-light reflectance imaging, fluorescence emission and near infrared imaging modalities.
In endoscopy, conventional white light imaging is typically used to view surface morphology, establish landmarks, and assess the internal organs based on appearance. Applications for viewing the respiratory and gastro-intestinal tracts are well established. Fluorescence imaging has evolved more recently and tissue autofluorescence has been exploited for the detection of early cancer. Similarly, observations of various native and induced chemical interactions, such as labeling tissue with proteins, for example, have been accomplished using fluorescence imaging. Fluorescently-tagged monoclonal antibodies are sometimes used to label specific cellular proteins, which in turn may be detected and/or be measured, optically.
Fluorescence imaging provides a means to detect disease while aiding in the determination of the boundaries that separate diseased from healthy tissue. Accordingly, these methods have been applied to the detection of early cancer in epithelial tissues. Except for the skin, epithelial tissue imaging is usually performed with an endoscope which provides access to the internal surfaces of various body organs such as the respiratory tract (lung) and GI tract. Tissue surfaces are usually not flat, and therefore the light distribution used to illuminate tissue and the light collection efficiency may vary markedly for different image pixels. To compensate for these conditions, and other variables associated with endoscopic imaging, normalization methods are employed to help correct for the geometrical and optical non-uniformities, ideally to make acquired images more diagnostically useful. Typically, this image normalization involves acquiring one image (a sort of reference), best matching (also called aligning or registering) it to a second (diagnostic image) and using the reference image it to correct or process one or more pixels of the diagnostic image. These endoscopic imaging methods are sometimes called two channel or multi-channel imaging. In modern devices, typically the images are acquired and manipulated in the digital domain and may be mixed, matched, colored or otherwise processed prior to presentation on a display device such as a monitor.
“Optical modulator” as used herein means a device or combination of optical and/or electro-optical devices used to alter the wavelength(s), and/or to alter the intensity, and/or to time-gate various spectra of electromagnetic radiation. Various filters, filter wheels, lenses, mirrors, micro-mirror arrays, liquid crystals, or other devices under mechanical or electrical control may be employed alone or in combination to comprise such an optical modulator. Certain embodiments of the present invention utilize two optical modulators, one associated with modulating light source spectrum used to interrogate (illuminate) a target object. A second optical modulator may be used to process the reflected and/or emitted light returned after interacting with the object. In some cases, such as in vivo endoscopic use, interaction of source illumination may be with lung tissue and returned light may include various reflected and re-emitted spectra.
Light in various spectra may be used to advantage. For example, near infrared light may be used to measure tissue oxygenation and may also help visualize or make measurements through blood. These properties may be used, for example, to verify that a biopsy was taken at the correct site. In addition, the present invention discusses, and in combination with existing spectral band imaging, exploits recently discovered tissue fluorescence properties in the near infrared spectral band.
2. Description of the Related Art
U.S. Pat. No. 6,364,829, to Fulghum, entitled, “Autofluorescence imaging system for endoscopy”, discusses a broad-band light source used to provide both visible light (which induces minimal autofluorescence) and ultraviolet light (capable of inducing tissue autofluorescence). Images are detected, for example, by a single imaging detector located at the distal tip of an endoscope. Electronic means are provided to switch (modulate) the source illumination spectrum used to interact with a target object, such as tissue. Various light sources, filter wheels, shutters, mirrors, dichroic mirrors, spectrum, light sources, intensities and timing diagrams are discussed in this prior art and are therefore included by reference, herein.
U.S. Pat. No. 6,148,227, to Wagnieres, entitled, “Diagnosis apparatus for the picture providing recording of fluorescing biological tissue regions”, discusses illumination spectrum and components for fluorescence imaging. In one embodiment red and green light components are directed to separate portions of a CCD detector with independent signal processing.
U.S. Pat. No. 6,061,591, to Freitag, entitled, “Arrangement and method for diagnosing malignant tissue by fluorescence observation”, discusses a strobed white-light illumination source and laser to stimulate fluorescence. Alternatively, a desired fluorescence spectrum may be isolated and provided from a single lamp, for example, a mercury-vapor xenon lamp. Filter wheels (with red, green and blue filters as well as filters to divide fluorescence into red and green components) and timing requirements are also discussed. Acquisition of white-light images and fluorescence images are performed in sequence, although both may be displayed on the monitor. Various Figures in '591 describe light sources which are similar to those contemplated for the present invention.
The system described in '591 provides the ability to switch back and forth between white light and fluorescence visualization methods, electronically, with display rates up to 10 Hz, or higher. Unlike other prior art (e.g. U.S. Pat. No. 5,647,368 which will be discussed), switching between normal visible light imaging, in full color, and fluorescence imaging is accomplished by an electronic switch rather than by physical optical modulation (switching) by the operator. The '591 patent also discusses a fluorescence excitation light at ultraviolet to deep violet wavelengths placed at the distal end of an endoscope, as well the use of gallium nitride laser diodes and mercury arc lamps for UV illumination of target objects which are also contemplated as illumination sources for various embodiments of the present invention. Also of interest, '591 discusses some limitations of endoscopes and more particularly limitations related to the UV-transmissive properties of optical fibers. Some of these limitations are addressed by co-pending U.S. application Ser. No. 10/226,406 to Ferguson/Zeng, filed approximately Aug. 23, 2002, entitled “Non-coherent fiber optic apparatus and imaging methods”.
U.S. Pat. No. 6,019,719, to Schulz, entitled, “Fully auotclavable electronic endoscope”, discusses an objective lens, crystal filter, IR filter and CCD chip arranged at the distal end of an endoscope for imaging.
U.S. Pat. No. 5,930,424 to Heimberger, entitled, “Device for connecting a fiber optic cable to the fiber optic connection of an endoscope”, discusses various aspects of coupling devices such as light sources to an endoscope.
U.S. Pat. No. 5,926,213 to Hafele, entitled, “Device for correcting the tone of color pictures recorded by a video camera”, such as an endoscope camera, is discussed along with a rotary transducer to activate tone correction. Color correction, calibration or normalization is useful for quantization from image data or comparison of images and is considered for various embodiments of the present invention.
U.S. Pat. No. 5,827,190, to Palcic, entitled, “Endoscope having an integrated CCD sensor”, discusses illumination light sources and sensors to measure various signals associated with tissue and tissue disease.
U.S. Pat. No. 5,647,368, to Zeng, entitled, “Imaging system for detecting diseased tissue using native fluorescence in the gastrointestinal and respiratory tract”, among other things discusses use of a mercury arc lamp to provide for white light and fluorescence imaging with an endoscope to detect and differentiate effects in abnormal or diseased tissue.
U.S. Pat. No. 5,590,660, to MacAulay, entitled, “Apparatus and method for imaging diseased tissue using integrated autofluorescence” discusses light source requirements, optical sensors, and means to provide a background image to normalize the autofluorescence image, for uses such as imaging diseased tissue.
U.S. Pat. No. 5,769,792, to Palcic, entitled, “Endoscopic imaging system for diseased tissue”, further discusses light sources and means to extract information from the spectral intensity bands of autofluorescence, which differ in normal and diseased tissue.
Also co-pending U.S. patent application Ser. No. 09/741,731, to Zeng, filed approximately Dec. 19, 2000 and entitled, “Methods and apparatus for fluorescence and reflectance imaging and spectroscopy and for contemporaneous measurements of electromagnetic radiation with multiple measuring devices”, (a continuation-in-part of U.S. Publication No. 2002/0103439) discusses contemporaneous methods of providing one mode of imaging and spectroscopy contemporaneously, but multiple imaging and associated spectroscopy modalities in sequential. In the present invention, methods are described to perform multimodal imaging contemporaneously at various desired wavelengths. Unlike Zeng's art, Zeng's present invention does not seek to provide images and measurements of wavelength spectrum, instead it seeks to provide contemporaneous multimodal imaging, where entire images in defined spectrum are detected and acquired for display and/or analysis.
U.S. Pat. No. 5,999,844, to Gombrich, entitled, “Method and apparatus for imaging and sampling diseased tissue using autofluorescence”, discusses a plurality of image detectors that receive excitation light as well as depositing biopsies in separate compartments or captive units.
U.S. Pat. No. 6,212,425, to Irion, entitled, “Apparatus for photodynamic diagnosis”, discusses endoscopic imaging using a light-induced reaction or intrinsic fluorescence to detect diseased tissue and delivery light for therapeutic use or to stimulate compounds that in turn provide therapy, for example.
U.S. Pat. No. 4,884,133, to Kanno, entitled “Endoscope light source apparatus”, discusses light sources, light guides and control of these elements for endoscopic use.
Endoscopes and imaging applications are further discussed in co-pending U.S. application Ser. No. 10/226,406 to Ferguson/Zeng, entitled “Non-coherent fiber optic apparatus and imaging methods”, which among other things, discusses apparatus to overcome some existing limitations of fiber optic devices, such as endoscopes.
U.S. Pat. No. 5,749,830 to Kaneko, entitled “Fluorescent endoscope apparatus”, discusses use of two light sources, a first (e.g. lamp) for white light and a second (e.g. helium-cadmium laser) for fluorescence, to provide interrogating spectra. Kaneko also employs a filter wheel placed in the pathway of a single detector. For multimodal imaging the filter wheel has a plurality of filters (e.g. three in FIGS. 4a and 5 in FIG. 4b). While they illustrate the display of two imaging modalities (110 of FIG. 7.), they do not discuss simultaneous real-time multimodal imaging. As this art discusses a wide range of issues utilized within the present invention, such as combining light sources, synchronization and filter wheels, (830) is included by reference herein.
Copending application by Zeng et al., filed on May 8, 2003 and entitled “Real time contemporaneous multimodal imaging and spectroscopy uses thereof”, is also included by reference.